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Lecture Than Students Who Are Prepared 


Abstract 

Students in three sections of introductory psychology, N = 1051, were asked about the utility of traditional, 
e.g. instructor, lectures and textbook, and nontraditional, e.g., clickers, podcasts and online lecture slides, 
teaching tools. Students who felt unprepared for college (25.9%) differed from their peers in their perceived 
utility of these tools. Both groups of students reported that novel teaching tools were less helpful than 
traditional teaching tools and while there was no group difference in the perceived usefulness of the novel 
tools, underprepared students found traditional teaching tools to be less helpful than did prepared students. 
When the individual tools were used to predict the amount of self-reported learning gains in these students, it 
was the traditional teaching tools that accounted for the greater proportion of variability in overall learning. 
These results suggest that, rather than adding new approaches to their teaching, instructors could best assist 
their underprepared students by helping them learn to make better use of traditional teaching tools. 
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Abstract 

Students in three sections of introductory psychology, N = 1051, were asked about the 
utility of traditional, e.g. instructor, lectures and textbook, and nontraditional, e.g., clickers, 
podcasts and online lecture slides, teaching tools. Students who felt unprepared for college 
(25.9%) differed from their peers in their perceived utility of these tools. Both groups of 
students reported that novel teaching tools were less helpful than traditional teaching tools 
and while there was no group difference in the perceived usefulness of the novel tools, 
underprepared students found traditional teaching tools to be less helpful than did prepared 
students. When the individual tools were used to predict the amount of self-reported 
learning gains in these students, it was the traditional teaching tools that accounted for the 
greater proportion of variability in overall learning. These results suggest that, rather than 
adding new approaches to their teaching, instructors could best assist their underprepared 
students by helping them learn to make better use of traditional teaching tools. 

Keywords: learning, innovation, academic readiness, technology 


Introduction 

The U.S. Department of Education projects that approximately one-half of the students who 
began a four-year bachelor's degree program in the fall of 2006 will actually receive their 
degree within six years. Projections appear even more dismal for students who were 
ranked among the bottom quarter of their high school classes, with only 20% expected to 
get their bachelor's degree or a two-year associates degree (Steinberg, 2010). As colleges 
and universities admit a greater diversity of students, this is the challenge educators face. 
Not all incoming college students will have received the same academic preparation in their 
high school educations. Estimates are that up to 55% of students entering 2- and 4-year 
institutions are underprepared for the challenges of higher education (Kay & Greenhill, 
2011). College instructors have attempted to assist these less than well-prepared students, 
going so far as to incorporate a variety of technologies in their instruction with the hope of 
improving learning (Aronowitz, 2011). 

These strategies are, in part, based on the principles of variation theory. According to 
variation theory, learners must experience new information from a variety of perspectives in 
order for learning to occur, as this facilitates new ways of learning (Oliver &Trigwell, 2005). 
The theory claims that without variation, there is no discernment; without discernment. 
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there is no learning (Marton, 2007). Thus, this theory suggests that using various forms of 
teaching media may help students discern important differences (variations) in patterns and, 
in turn, enhance learning. In response to this theory, colleges and universities are doing 
just that: including various technologies and innovations in classrooms (Twig, 2000). 

Introducing new technologies into the classroom as a means of enhancing learning is not a 
new notion. Since the 1920s, we have seen a push for media use in an educational setting 
(e.g., radio, television, and computer programs) because of the strong claims for pedagogic 
value, yet few have proven to be effective (Cuban, 1986). However, as more colleges and 
universities incorporate technology into teaching, such as the use of personal response 
systems ("clickers") and podcasts, it behooves us to examine just how effective these 
strategies are and whether they are providing enhanced learning opportunities for 
underprepared students. 

Clickers are one such instructional technology that have recently become popular and are 
being used to promote student-instructor interaction in large lectures. Survey-based 
studies have found that both students and teachers rate clickers to be helpful as well as 
enjoyable to use (Beekes, 2006; Draper & Brown, 2004; Hatch, Jensen, & Moore, 2005; 
Latessa & Mouw, 2005; Zahorik, 1996). Consistent with these findings, a recent quasi¬ 
experiment concluded that, "if the goal is to help students learn in large college lecture 
classes, there is reason to consider using a personal response system to foster student- 
instructor interaction during class" (Mayer et al., 2009). While user feedback has been 
positive, evidence is mixed as to whether these devices increase student performance 
(Lantz, 2010). 

Podcasting, another relatively new teaching tool, allows students to listen to recorded 
lectures; however, results are mixed as to whether or not it is more beneficial than 
traditional tools, such as the textbook or coming to lecture. Some studies have found that 
students prefer supplementary material in the podcasting format and rated podcasts as at 
least as useful as printed handouts or the textbook (Copley, 2007; Evans, 2008). Copley 
(2007) also found that podcasts provided better learning outcomes for students as they 
facilitate better note revision and exam preparation, more student engagement in lecture 
topics, and the flexibility for students to learn complex material at their own pace. However, 
while students enjoy the flexibility of podcasting, traditional lectures are still preferred by 
students as a means of learning new information (Stephenson, Brown, & Griffin, 2008). 

The present study examines the self-reports of 3 sections of students on an end of semester 
course evaluation. The Student Assessment of Learning Gains (SALG; Seymour, Weise, 
Hunter, & Daffinrud, 2000) allows instructors to get feedback from students about their 
learning gains in a course and the perceived utility of different teaching and learning tools. 

We investigated whether students who differed in their perceived level of preparation for 
college differed in their ratings of the utility of traditional teaching tools, e.g. lectures, 
textbook, pace of class, and instructor, versus novel or nontraditional teaching tools, e.g., 
podcasts, clickers, and online resources. 

We predicted that the students who felt prepared for college by their high school education 
would rate the traditional tools to be more useful than nontraditional tools, and that 
prepared students would find these tools were more useful than underprepared students. 

We expected the opposite pattern of results for underprepared students, i.e., to report 
higher ratings of utility for the nontraditional tools than prepared students and to rate the 
nontraditional tools as more helpful than the traditional tools. 
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Method 


Participants 

Participants were 1,051 students enrolled across three sections of introductory psychology 
at a large Midwestern university. The majority of participants were female (54.7%), 
Caucasian (85.8%), and in their first year of college (77.2%). Of the 1,051 enrolled 
students, 924 (87.9%) completed the SALG (Seymour et al. 2000). Response rates were 
comparable across sections, and reflecting the class demographics, respondents were 
primarily Caucasian (85.2%) and in their first year of college (75.7%).' 

Procedure 

During the last week of the semester, students were asked to complete the SALG, for which 
they received course credit. This assignment was 1 of 10 assigned over the course of the 
semester. While course credit was an incentive to complete the assignments, students were 
allowed to miss 2 of these assignments and still receive full credit. All responses to the 
survey were anonymous. Students logged on to the SALG site using their campus email 
addresses, but this information was kept separate from and could not be linked to their 
survey responses. 

Teaching Tools 

Traditional Tools - The seven traditional teaching tools evaluated by the students were 
lectures, textbook, instructor, in-class demonstrations, discussions in class, pace of the 
course, and how all assignments, readings, and activities fit together. Classes met twice a 
week for 75 minutes over the course of a 15 week semester. The textbook for the course 
was a standard "brief" psychology textbook, Essentials of Psychology (Bernstein, Nash, 
Clarke-Stewart, Penner, & Roy, 2008), and the class covered a chapter's worth of material 
every week. In-class demonstrations consisted of occasional demonstrations of concepts 
such as neuronal communication, conditioning, and memory. 

Nontraditional Tools - The nontraditional tools consisted of podcasts, clickers, and online 
resources. There were more than 60 podcasts ranging from 2 to 38 minutes in length. The 
median duration was 7 minutes and 12 seconds. Average duration was 8 minutes and 13 
seconds, SD = 4:53. Podcasts were recommended but not required and were intended to 
expand upon topics covered in lecture. Clickers were used in every class, during which, the 
professor would ask between two and ten questions per class (M = 4). The questions were 
variably distributed throughout the lecture and primarily consisted of concept review of 
material covered earlier in that class or of material covered in previous lectures. Other 
questions were designed to demonstrate psychological concepts, e.g., availability heuristic, 
anchoring and adjustment, and critical thinking exercises that required students to predict 
outcomes of psychological studies. 

Online resources consisted of lecture slides, chapter review and exam review quizzes, the 
textbook website, a bank of frequently asked questions (FAQs), and course related web 
links. The lecture slides were pdf handouts of the PowerPoint slides with space for notes. 
The content of the slides was also provided in outline form in Word formatted documents. 
The chapter-review quizzes consisted of 10 multiple choice questions randomly drawn from 
a larger bank of questions on each topic covered in that section. These were required for 
one section and recommended for the other two sections. The exam review quizzes 
selected 20 questions from across topics for each exam and were designed to allow students 
to test their preparedness for each exam. Students were allowed to take both the chapter 
review quizzes and the exam review quizzes an unlimited number of times. There were 
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more than 75 FAQs covering topics that include test taking, studying strategies, and 
questions about course material. The course related web links provided students with 
videos, outside resources to support individual interests, scholarly articles, and an RSS feed 
of related articles in the popular press. 

Measures 

The SALG (Seymour et al., 2000) is a modifiable online instrument that allows students to 
provide feedback to their instructors about various aspects of their course. In this study, 
students were asked to rate the utility of various traditional and novel teaching tools and to 
rate their learning gains in specific course content areas as well as their gains in more 
broadly defined areas such as critical thinking, confidence in their abilities, and enthusiasm 
for the material. The rating scales for utility of tools ranged from 1 to 5 (No Help - Great 
Help). Learning gains were also rated from 1 to 5 (No Gains - Great Gains). Additional 
questions at the end of the instrument asked students about their race/ethnicity, whether it 
was their first year in school (Yes/No), and whether they felt that their high school 
education had sufficiently prepared them for college (Yes/No). Average Learning (AL) was 
computed by averaging the amount of self-reported learning gains across topic areas, which 
included research methods, the brain, sleep and hypnosis, learning, memory, cognition, 
emotion, personality, psychological disorders, and social psychology. Overall internal 
consistency for this composite measure was .87 (Cronbach's alpha for each individual 
section ranged from .80 - .90). The measure of Learning Gains (LG) was a composite score 
of students' self-reported learning gains on 13 items pertaining to their interest and 
enthusiasm for the material, increase in skills in the subject area, integration of learning, 
and understanding the relationships between different psychological concepts. Cronbach's 
alpha for this measure of LG was .95 (range .95 - .96). Students rated the usefulness of six 
traditional teaching tools (lectures, textbook, instructor, in-class demonstrations, pace of 
the course, and how all assignments, readings, and activities fit together) and seven novel 
teaching tools (podcasts, clickers, and online resources, such as lecture slides, review 
questions, the textbook website, frequently asked questions, and course related web links). 
Across the three sections, the internal consistency for traditional teaching tools was .81 
(range .77 - .84). For novel teaching tools, overall internal consistency was .83 (range 
.82 - .85). 


Results 

Of the students surveyed, 238 (25.9%) reported that they felt that their high school 
education had left them unprepared for college. 

Average Learning and Learning Gains 

Students' AL and LG were examined by way of separate two-way multivariate analysis of 
variances (MANOVA) with Section and Preparation (Prepared/Unprepared) as the between 
subjects variables. 

These analyses revealed a main effect for Preparation, F(2, 912) = 3.09, p = .046, but no 
significant differences between Sections, F(4, 1826) = 0.55, p = .693, or any significant 
Section X Preparation interaction, F(4, 1826) = 0.34, p = .854. Students who reported that 
they were inadequately prepared for college reported lower amounts of AL, F(l, 913) = 
6.00, p = .014, partial eta squared = .006, and lower levels of LG, F(l, 913) = 3.87, p 
= .049, partial eta squared = .005 (see Table 1). 
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Table 1. Average Learning Gains and Utility of Teaching Tools for Prepared and Unprepared Students 
(SD in parentheses) 



Section 1 
Prepared 

Section 2 
Prepared 

Section 3 
Prepared 

Total Sample 
Prepared 


Yes 

n = 235 

No 

n = 66 

Yes 

n = 235 

No 

n = 79 

Yes 

n = 211 

No 

n = 93 

Yes 

n = 681 

No 

n = 238 

Average 
Learning (AL) 

4.22 

(.54) 

4.11 

(.54) 

4.14 

(.71) 

4.04 

(.77) 

4.22 

(.58) 

4.07 

(.76) 

4.19 

(.62) 

4.07 

(.71) 

Learning 

Gains (LG) 

3.81 

(.79) 

3.71 

(.80) 

3.75 

(.88) 

3.57 

(.90) 

3.77 

(.88) 

3.66 

(.92) 

3.78 

(.85) 

3.65 

(.88) 

Traditional 

Tools 

3.99 

(.60) 

3.87 

(.61) 

3.96 

(.68) 

3.78 

(.76) 

3.99 

(.66) 

3.88 

(.71) 

3.97 

(.64) 

3.84 

(.70) 

Novel Tools 

3.24 

(.73) 

3.26 

(.74) 

3.29 

(.77) 

3.35 

(.78) 

3.37 

(.78) 

3.25 

(.79) 

3.30 

(.76) 

3.28 

(.74) 


Teaching Tools 

The utility of the traditional and novel teaching tools was examined by way of a three-way 
repeated measures MANOVA. Section and Preparation were the between subjects variables 
and Tool (Traditional/Novel) was the within subjects variable. This revealed a main effect for 
Tool, F( 1, 913) = 568.30, p < .001, partial eta squared = .384, which was the result of 
traditional teaching tools being rated as more useful than the novel teaching tools by all 
students. Furthermore, there was a significant Preparation X Tool interaction, F(l, 913) = 
5.23, p < .03, partial eta squared = .006 but no other main effects or interactions were 
significant, all F's < 2.50, all ps > .10. Separate ANOVAs for traditional and novel teaching 
tools were computed comparing prepared and unprepared students collapsed across the 
three sections in order to decompose the significant interaction. There was no difference 
between prepared and unprepared students in their ratings of the utility of novel teaching 
tools, F(l, 918) = 0.06, p > .80, partial eta squared = .000, M = 3.30 and M = 3.28 for 
each group respectively. There was a significant difference in how these students rated the 
traditional teaching tools, F(l, 918) = 7.16, p < .01, partial eta squared = .008, and this 
was a result of underprepared students rating the traditional teaching tools lower than the 
prepared students, M = 3.84 and M = 3.97 respectively, (see Figure 1). 
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Professor 

Lecture 

Instructional Approach 
Textbook 

How Topics, Activities Fit Together 
Class Demonstrations 
Pace of Class 
Class Discussion 
Course Related Links 
Frequently Asked Questions 
Textbook Website 
Clicker 
Podcasts 
Online Lecture Slides 
Online Review Quizzes 


0 0.5 1 1.5 2 2.5 3 3.5 4 4.5 5 

■ Prepared ■Unprepared 


Figure 1. Reported utility of individual traditional and nontraditional teaching tools by students 
who felt prepared and unprepared for college by their high school education. 


Predicting Students' Learning Gains 

In order to examine which individual teaching tools accounted for the most variability in 
students' LG, exploratory hierarchical regression analyses were computed separately for 
each group. In each analysis, section was entered into the model first to control for any 
systematic differences between classes. The individual teaching tools were entered in 
stepwise hierarchical fashion according to the amount of variance accounted for by each tool. 
For prepared students, eight teaching tools accounted for 54.9% of the total variance in LG. 
In order of proportion of variance accounted for, these were the Professor, the Way Topics, 
Activities, Readings and Assignments Fit Together, the Pace of the Class, Demonstrations in 
Class, Frequently Asked Questions, Lectures, Textbook, and Course Related Links. For 
unprepared students, seven teaching tools accounted for 58.9% of the total variance in AL. 

In order of proportion of variance accounted for, these were the Professor, the Way Topics, 
Activities, Readings and Assignments Fit Together, the Clicker, the Pace of the Class, Online 
Quizzes, Class Discussion, and the Frequently Asked Questions. (See Table 2). 
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Table 2. Multiple Regression Predicting Learning Gains (LG) in Prepared and Unprepared Students 



Variables Entered 

R 

R Square 

Significance 

F Change 

Prepared 

Students 

Section 

.022 

0 

.565 


Professor 

.577 

.333 

.000 


Way fit together 

.662 

.438 

.000 


Pace of class 

.700 

.490 

.000 


Demonstrations 

.718 

.516 

.000 


Frequently Asked Questions 

.727 

.539 

.000 


Lectures 

.733 

.537 

.001 


Textbook 

.738 

.545 

.000 


Course Related Links 

.741 

.549 

.016 

Unprepared 

Students 

Section 

.019 

0 

.771 


Professor 

.608 

.370 

.000 


Way fit together 

.685 

.469 

.000 


Clicker questions 

.718 

.515 

.000 


Pace of class 

.735 

.540 

.001 


Review Quizzes 

.753 

.567 

.000 


Class Discussion 

.761 

.580 

.010 


Frequently Asked Questions 

.767 

.589 

.027 


Discussion 

Students who felt prepared for college reported greater AL and LG than students who 
reported less college preparation by their high school, providing a measure of validity to our 
construct of readiness for college. The differences in the reported utility of novel and 
traditional teaching tools between the two groups of students were unexpected. While we 
had expected that traditional teaching tools would be rated more highly by college prepared 
students, we had predicted that underprepared students would rate the nontraditional tools 
more highly. What we found was that both groups perceived traditional teaching tools as 
being more effective than nontraditional or novel tools. While there was no difference in the 
reported utility of the novel tools between groups, the two groups of students did differ in 
their evaluation of the traditional teaching tools. Students who felt prepared for college 
reported that these tools were of greater utility than underprepared students. 

While traditional teaching tools were perceived as less helpful by underprepared students as 
compared to students who felt prepared for college, these traditional tools were significant 
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predictors of overall learning gains for both prepared and underprepared students. For both 
groups of students, the pace of class, the way topics, activities, and readings fit together 
and the professor all played significant roles in their overall learning gains. These three 
variables alone accounted for almost 50% of the variance in overall student learning gains. 
While the number of nontraditional teaching tools that were significant predictors was 
greater than the number of nontraditional tools that predicted the learning gains of 
prepared students, traditional teaching tools accounted for a greater proportion of learning 
gains in both groups of students. These findings suggest that while innovation can be useful 
in the classroom, it is traditional teaching tools that are most critical to students' learning. 
The fact that these traditional tools are perceived as less helpful by underprepared students 
suggests that colleges and universities need to provide incoming students with instruction 
on how to make best use of these critical resources. 

There are some limitations to the current study which should be addressed in future 
research. Underprepared students were identified on the basis of their own self-report, and 
the validity of that judgment cannot be determined in the current research. For instance, it 
could be the case that students were judging their level of preparedness based simply on 
whether they had performed as well as they had expected in this particular class. It would 
be informative to explore how exactly students felt unprepared for college, are there 
particular skills such as note-taking or test preparation that they feel they are lacking or is 
their lack of preparedness related to a particular area of study, such as mathematical skills. 
It would also be useful to find out if these students felt unprepared in all of their courses or 
only in particular classes. Another limitation of the current study is that students evaluation 
of the utility of the traditional and nontraditional teaching tools was purely subjective. It 
could be the case that certain teaching tools had a greater impact on the students' learning 
than they in fact realized. An examination of changes in classroom performance in response 
to the introduction or removal of different teaching tools would provide greater support for 
the initial findings reported in this paper. Similarly, it would be helpful if we knew students' 
actual performance in the class rather than having to rely on their expectation of their grade 
in the class. 

While we might argue that our results suggest that faculty could better help underprepared 
students be successful by providing instruction on how to make use of these traditional 
learning tools rather than introducing novel tools and approaches into their classes, it could 
be argued that greater instruction on the use of novel teaching tools, as well, would result 
in a more substantial impact of these new tools on student learning for all students. We 
should note that there were no questions asked of students addressing the extent to which 
they felt adequately informed on how to make use of the different teaching tools, and their 
reported utility may simply reflect their level of comfort with the different instructional tools. 
As further exploration of this topic continues, we would do well to consider the comments of 
Mayer et al. (2009) who note that, "The search for an appropriate educational technology 
can become a misleading and potentially unproductive adventure...." 
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Footnotes 

1 Students were not asked to identify their gender on the SALG 
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